Rank by likeCount Ratio¶

Video Ranking Model

What we're really doing:

$$p_i \text{ = Probability that an individual viewer will like video }i\text{,}$$ $$L_i \text{ = # of Likes for video }i\text{,}$$ $$D_i \text{ = # of Dislikes for video }i$$ $$n_i = L_i + D_i$$

Model Realistic? Sort of.

Maximum Likelihood Estimate for $p_i$ = $\frac{L_i}{n_i}$

Good estimate? Yea, but not when the sample size (i.e. $n_i$) is small (e.g. 1/1, 2/2, 0/1)

How can we model the probability of a like but somehow account for sample size?¶

The Beta Distribution

It's only defined on 0 to 1 and has two parameters, plenty enough to manipulate the shape to be what we want.

Probability Density Function:

$$p(x) = \frac{x^{\alpha - 1}(1 - x)^{\beta - 1}}{B(\alpha, \beta)},$$ where $B$ is the Beta Function

$p_i$

Making up a Solution

Let's just say this because why not:

• $\alpha$ parameter in $B$ distribution = # video likes
• $\beta$ parameter in $B$ distribution = # video dislikes

What do these distributions look like for videos with the same like / dislike ratio?

Say 4 videos had the following stats:

Video	Likes	Dislikes	Ratio
1	9	3	9/3 = 3
2	15	5	15/5 = 3
3	48	16	48/16 = 3
4	600	200	600/200 = 3

How to get there?¶

BEFORE we see any data, let's say we estimate $p_i$ to be something flat like this:

Bayes Theorem¶

Given a prior belief in $p_i$ AND some data $D$, what should be our new belief as to what $p_i$ is?

Luckily there are some rules that will help:

$$Pr(p_i|D) = \frac{Pr(D|p_i) \cdot Pr(p_i)}{Pr(D)}$$

In English:

$$\text{The distribution of }p_i\text{ based on data AND previous beliefs}$$

$$= \frac{Pr( \text{X likes and Y dislikes given a value for }p_i) \cdot Pr(p_i\text{in the first place})}{\text{the sum of all possible values for the numerator}}$$

The bayesian lingo:

$$\text{posterior} = \frac{\text{likelihood} \cdot \text{prior}}{\text{marginal}}$$

Example¶

• Say our data, $D$, was 2 likes and 1 dislike
• Also say the we want to calculate an approximation to the posterior for $p \in [.1, .5, .9]$

p	Pr(D|p)	Pr(p)	Pr(D|p) * Pr(p)
.1	3 * .1^2 * (1-.1) = 0.027	1	0.027
.5	3 * .5^2 * (1-.5) = 0.375	1	0.375
.9	3 * .9^2 * (1-.9) = 0.243	1	0.243

marginal - The sum of all "likelihood * prior" values = 0.027 + 0.375 + 0.243 = 0.645

Back to Theory¶

• The "likelihood" or $Pr(D|p)$ above is technically from a binomial distribution

  • The mass function of this distribution is:

    $$P(k, n) \sim \textstyle {n \choose k}\, p^k (1-p)^{n-k}$$
    • $k$ is the number of likes in our case
    • $n$ is the number of likes + number of dislikes (ie votes)
    • $p$ is a current estimate for the actual $p_i$ (ie the probability of getting a like)

• Recall the "prior" was from a Beta distribution with density function:

  $$P(p) = p^{\alpha - 1}(1 - p)^{\beta - 1}/B(\alpha, \beta)$$
  • $p$ is probability of getting a like
  • Our prior used so far was $Beta(1, 1)$ but in general, you can have any prior like $Beta(\alpha, \beta)$

Posteriors¶

• The posterior actually looks like this then:
  
  
  

$$Posterior = \frac{Binomial \text{ (our likelihood)} \cdot Beta \text{ (our prior)}}{\text{Integral over all possible values above}}$$

• Given the above things, plugging it all together gives (but leaving out the sum/integral on the bottom):
  
  
  

$$Pr(p_i|L_i,D_i,n_i) \propto \text{Binomial}(p_i, L_i, n_i) \cdot \text{Beta}(\alpha, \beta) = \frac{ {n_i \choose L_i}\, p_i^{L_i} (1-p_i)^{D_i} p_i^{\alpha - 1}(1 - p_i)^{\beta - 1}}{B(\alpha, \beta)}$$

The Punchline¶

Through algrebra and magic, the posterior actually reduces to another BETA distribution. The final form is:

$$Pr(p_i|L_i,D_i,n_i) = \text{Beta}(\alpha + L_i, \beta + D_i)$$

If our prior was uniform (ie Beta(1, 1)), what is the distribution of $p_i$ if a video has 5 likes and 2 dislikes?

$$p_1 \sim \text{Beta}(1 + 5, 1 + 2) = \text{Beta}(6, 3)$$

And for one with 50 likes and 20 dislikes?

$$p_2 \sim \text{Beta}(1 + 50, 1 + 20) = \text{Beta}(51, 21)$$

Applying Bayesian Rank to Data¶

• Apply the model above to our YouTube videos and see how that changes the list:

BREAK

How to Probably Not Die on Vacation¶

Forecasting Homicide Rates in the Caribbean¶

Statement from 2013 United Nations Office of Drugs and Crime:

"Caribbean countries are home to to 8.5% of the world's population, but they account for about 27% of world crime"

Raw Caribben Homicide Data¶

• Data taken from a study done by the United Nations Office on Drug and Crime (in 2013)
• No data present past 2013, and a lot of it is missing for other years in some countries too
• This is almost definitely the most reputable, up-to-date source of data like this out there

What does this mean for 2015?

Switch to R¶

Mixture Models¶

Making a Paint By Numbers¶

• Images are really just arrays
• 2 dimensional arrays with an R, G, and B color value at each location
• So really 3 dimensional

An example:

Convert Raw Image¶

• Need to put into "LAB" space instead of RGB
• Also need to flatten into a data frame

Clusters Inferred¶

Give each "predicted" cluster a random color:

Final Result¶

Getting the edges is a little tricky, but the clustering still does the bulk of the work

Mixture Models in a Real World Setting¶

• Mixtures are often used to infer clusters of behaviors in data from anything:
  • Motion Detection - Detecting types of movements from motion sensors
  • Political Partisanship - Determining democrat vs republican using survey responses
  • Topic Modeling - Determining which documents relate to which known topics

Old News .. who cares, there are better, domain-specific ways to do most of these things anyhow.

Having to know or guess at the number of clusters is a huge downer ..

Dirichlet Process Applications¶

• Topic Modeling - When you don't know how many topics there are in a corpus of text
• Motion Detection - Determining groups of movements like dancing, running, jumping from motion sensors without knowing they're coming
• MRI Activation Modeling - Determining clusters of brain activation during sensory tasks
• Speaker Diarisation - Figuring out who's speaking in a room with an unknown number of people
• Variable Selection - The "Dirichlet Lasso" for selecting groups of features (as opposed to group lasso)

Combining Hierarchichal Modeling, Infinite Mixtures, and Gaussian Processes:

Done